Temporal attention networks for biomedical hypothesis generation.

OBJECTIVES: Hypothesis Generation (HG) is a task that aims to uncover hidden associations between disjoint scientific terms, which influences innovations in prevention, treatment, and overall public health. Several recent studies strive to use Recurrent Neural Network (RNN) to learn evolutional embeddings for HG. However, the complex spatiotemporal dependencies of term-pair relations will be difficult to depict due to the inherent recurrent structure. This paper aims to accurately model the temporal evolution of term-pair relations using only attention mechanisms, for capturing crucial information on inferring the future connectivities. METHODS: This paper proposes a Temporal Attention Networks (TAN) to produce powerful spatiotemporal embeddings for Biomedical Hypothesis Generation. Specifically, we formulate HG problem as a future connectivity prediction task in a temporal attributed graph. Our TAN develops a Temporal Spatial Attention Module (TSAM) to establish temporal dependencies of node-pair (term-pair) embeddings between any two time-steps for smoothing spatiotemporal node-pair embeddings. Meanwhile, a Temporal Difference Attention Module (TDAM) is proposed to sharpen temporal differences of spatiotemporal embeddings for highlighting the historical changes of node-pair relations. As such, TAN can adaptively calibrate spatiotemporal embeddings by considering both continuity and difference of node-pair embeddings. RESULTS: Three real-world biomedical term relationship datasets are constructed from PubMed papers. TAN significantly outperforms the best baseline with 12.03%, 4.59 and 2.34% Micro-F1 Score improvement in Immunotherapy, Virology and Neurology, respectively. Extensive experiments demonstrate that TAN can model complex spatiotemporal dependencies of term-pairs for explicitly capturing the temporal evolution of relation, significantly outperforming existing state-of-the-art methods. CONCLUSION: We proposed a novel TAN to learn spatiotemporal embeddings based on pure attention mechanisms for HG. TAN learns the evolution of relationships by modeling both the continuity and difference of temporal term-pair embeddings. The important spatiotemporal dependencies of term-pair relations are extracted based solely on attention mechanism for generating hypotheses.

Learning temporal difference embeddings for biomedical hypothesis generation.

MOTIVATION: Hypothesis generation (HG) refers to the discovery of meaningful implicit connections between disjoint scientific terms, which is of great significance for drug discovery, prediction of drug side effects and precision treatment. More recently, a few initial studies attempt to model the dynamic meaning of the terms or term pairs for HG. However, most existing methods still fail to accurately capture and utilize the dynamic evolution of scientific term relations. RESULTS: This article proposes a novel temporal difference embedding (TDE) learning framework to model the temporal difference information evolution of term-pair relations for predicting future interactions. Specifically, the HG problem is formulated as a future connectivity prediction task on a temporal sequence of a dynamic attributed graph. Our approach models both the local neighbor changes of the term-pairs and the changes of the global graph structure over time, learning local and global TDE of node-pairs, respectively. Future term-pair relations can be inferred in a recurrent network based on the local and global TDE. Experiments on three real-world biomedical term relationship datasets show the effectiveness and superiority of the proposed approach. AVAILABILITY AND IMPLEMENTATION: The data and source codes related to TDE are publicly available at https://github.com/Huiweizhou/TDE. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

A new feature selection method based on feature distinguishing ability and network influence.

The occurrence and development of diseases are related to the dysfunction of biomolecules (genes, metabolites, etc.) and the changes of molecule interactions. Identifying the key molecules related to the physiological and pathological changes of organisms from omics data is of great significance for disease diagnosis, early warning and drug-target prediction, etc. A novel feature selection algorithm based on the feature individual distinguishing ability and feature influence in the biological network (FS-DANI) is proposed for defining important biomolecules (features) to discriminate different disease conditions. The feature individual distinguishing ability is evaluated based on the overlapping area of the feature effective ranges in different classes. FS-DANI measures the feature network influence based on the module importance in the correlation network and the feature centrality in the modules. The feature comprehensive weight is obtained by combining the feature individual distinguishing ability and feature influence in the network. Then crucial feature subset is determined by the sequential forward search (SFS) on the feature list sorted according to the comprehensive weights of features. FS-DANI is compared with the six efficient feature selection methods on ten public omics datasets. The ablation experiment is also conducted. Experimental results show that FS-DANI is better than the compared algorithms in accuracy, sensitivity and specificity on the whole. On analyzing the gastric cancer miRNA expression data, FS-DANI identified two miRNAs (hsa-miR-18a* and hsa-miR-381), whose AUCs for distinguishing gastric cancer samples and normal samples are 0.959 and 0.879 in the discovery set and an independent validation set, respectively. Hence, evaluating biomolecules from the molecular level and network level is helpful for identifying the potential disease biomarkers of high performance.

Improving the recall of biomedical named entity recognition with label re-correction and knowledge distillation.

BACKGROUND: Biomedical named entity recognition is one of the most essential tasks in biomedical information extraction. Previous studies suffer from inadequate annotated datasets, especially the limited knowledge contained in them. METHODS: To remedy the above issue, we propose a novel Biomedical Named Entity Recognition (BioNER) framework with label re-correction and knowledge distillation strategies, which could not only create large and high-quality datasets but also obtain a high-performance recognition model. Our framework is inspired by two points: (1) named entity recognition should be considered from the perspective of both coverage and accuracy; (2) trustable annotations should be yielded by iterative correction. Firstly, for coverage, we annotate chemical and disease entities in a large-scale unlabeled dataset by PubTator to generate a weakly labeled dataset. For accuracy, we then filter it by utilizing multiple knowledge bases to generate another weakly labeled dataset. Next, the two datasets are revised by a label re-correction strategy to construct two high-quality datasets, which are used to train two recognition models, respectively. Finally, we compress the knowledge in the two models into a single recognition model with knowledge distillation. RESULTS: Experiments on the BioCreative V chemical-disease relation corpus and NCBI Disease corpus show that knowledge from large-scale datasets significantly improves the performance of BioNER, especially the recall of it, leading to new state-of-the-art results. CONCLUSIONS: We propose a framework with label re-correction and knowledge distillation strategies. Comparison results show that the two perspectives of knowledge in the two re-corrected datasets respectively are complementary and both effective for BioNER.

PEGylated DOX-coated nano graphene oxide as pH-responsive multifunctional nanocarrier for targeted drug delivery.

Nano graphene oxide (NGO) has high drug-loading capacity due to its huge surface area. However, the limited stability and the poor biocompatibility of NGO hampered its application as drug delivery carrier under physiological conditions. Thereby, a new strategy of using chemical conjugation on NGO with hydrophilic polymers was adopted but currently was too complicated, low yield and costly. In this study, doxorubicin-hyd-PEG-folic acid (DOX-hyd-PEG-FA) polymers were coated on the surface of NGO via π-π stocking and the hydrophobic effect between DOX and NGO. With the PEG shell protection, the biocompatibility of NGO was significantly improved. The drug-loading capacity of nanoparticles was more than 100%. FA ligands on the nanoparticle could guide the nanoparticles actively targeting to tumour cells. The hydrazone bond between DOX and PEG was decomposed spontaneously in the weakly acidic environment, which made PEG layer dissociated from NGO. Furthermore, DOX was easily protonized at low pH conditions, which weakened the interaction between DOX and NGO. Thus, DOX could be released rapidly from the nanoparticles in tumour cells. In summary, NGO@DOX-hyd-PEG-FA is an easy-prepared nanoparticle with excellent biocompatibility, high pH-sensitivity and active tumour targeting. Therefore, it is a promising multifunctional nanocarrier effective for targeted drug delivery.

Chinese adolescents' mindfulness and internalizing symptoms: The mediating role of rumination and acceptance.

BACKGROUND: Internalizing problems during adolescence are common psychiatric symptoms. Previous research has demonstrated that mindfulness was significantly and negatively associated with and mindfulness-based interventions would be efficacious for aiming at adolescents' internalizing problems. However, research about how mindfulness could improve internalizing symptoms among Chinese adolescents is sparse and its potential mechanism is still unclear. The present study adopted rumination and acceptance to examine their mediation effects between Chinese adolescents' mindfulness and internalizing symptoms. METHODS: A final sample of 1,554 adolescents (aged from 10 to 18 years old, M = 15.58, SD =1.25) were recruited from schools in South China. Participants were asked to complete a package of questionnaires measuring mindfulness, internalizing problems (indicated by generalized anxiety and depression), rumination, and acceptance. RESULTS: Structural equation model confirmed our hypothesis and showed that rumination, as well as acceptance, significantly mediated the relationship between Chinese adolescents' mindfulness level and internalizing symptoms (generalized anxiety and depression). LIMITATIONS: a) only cross-sectional design was employed in the study; b) most of participants were normal adolescent students, without a diagnosis of any psychiatric disorder; c) all the measures were self-reported by adolescents. CONCLUSIONS: Mindfulness not only directly impacted on adolescents' internalizing problems, but also indirectly improved their anxious and depression emotions via the reduction of rumination and the increase of acceptance. Hence, mindfulness training as well as the application of emotion regulation skills may be useful for adolescents exposed to the likelihood of suffering from internalizing problems.

Knowledge-enhanced biomedical named entity recognition and normalization: application to proteins and genes.

BACKGROUND: Automated biomedical named entity recognition and normalization serves as the basis for many downstream applications in information management. However, this task is challenging due to name variations and entity ambiguity. A biomedical entity may have multiple variants and a variant could denote several different entity identifiers. RESULTS: To remedy the above issues, we present a novel knowledge-enhanced system for protein/gene named entity recognition (PNER) and normalization (PNEN). On one hand, a large amount of entity name knowledge extracted from biomedical knowledge bases is used to recognize more entity variants. On the other hand, structural knowledge of entities is extracted and encoded as identifier (ID) embeddings, which are then used for better entity normalization. Moreover, deep contextualized word representations generated by pre-trained language models are also incorporated into our knowledge-enhanced system for modeling multi-sense information of entities. Experimental results on the BioCreative VI Bio-ID corpus show that our proposed knowledge-enhanced system achieves 0.871 F1-score for PNER and 0.445 F1-score for PNEN, respectively, leading to a new state-of-the-art performance. CONCLUSIONS: We propose a knowledge-enhanced system that combines both entity knowledge and deep contextualized word representations. Comparison results show that entity knowledge is beneficial to the PNER and PNEN task and can be well combined with contextualized information in our system for further improvement.

Improving neural protein-protein interaction extraction with knowledge selection.

Protein-protein interaction (PPI) extraction from published scientific literature provides additional support for precision medicine efforts. Meanwhile, knowledge bases (KBs) contain huge amounts of structured information of protein entities and their relations, which can be encoded in entity and relation embeddings to help PPI extraction. However, the prior knowledge of protein-protein pairs must be selectively used so that it is suitable for different contexts. This paper proposes a Knowledge Selection Model (KSM) to fuse the selected prior knowledge and context information for PPI extraction. Firstly, two Transformers encode the context sequence of a protein pair according to each protein embedding, respectively. Then, the two outputs are fed to a mutual attention to capture the important context features towards the protein pair. Next, the context features are used to distill the relation embedding by a knowledge selector. Finally, the selected relation embedding and the context features are concatenated for PPI extraction. Experiments on the BioCreative VI PPI dataset show that KSM achieves a new state-of-the-art performance (38.08 % F1-score) by adding knowledge selection.

Knowledge-aware attention network for protein-protein interaction extraction.

Protein-protein interaction (PPI) extraction from published scientific literature provides additional support for precision medicine efforts. However, many of the current PPI extraction methods need extensive feature engineering and cannot make full use of the prior knowledge in knowledge bases (KBs). KBs contain huge amounts of structured information about entities and relationships, therefore play a pivotal role in PPI extraction. This paper proposes a knowledge-aware attention network (KAN) to fuse prior knowledge about protein-protein pairs and context information for PPI extraction. The proposed model first adopts a diagonal-disabled multi-head attention mechanism to encode context sequence along with knowledge representations learned from KBs. Then a novel multi-dimensional attention mechanism is used to select the features that can best describe the encoded context. Experiment results on the BioCreative VI PPI dataset show that the proposed approach could acquire knowledge-aware dependencies between different words in a sequence and lead to a new state-of-the-art performance.

Knowledge-guided convolutional networks for chemical-disease relation extraction.

BACKGROUND: Automatic extraction of chemical-disease relations (CDR) from unstructured text is of essential importance for disease treatment and drug development. Meanwhile, biomedical experts have built many highly-structured knowledge bases (KBs), which contain prior knowledge about chemicals and diseases. Prior knowledge provides strong support for CDR extraction. How to make full use of it is worth studying. RESULTS: This paper proposes a novel model called "Knowledge-guided Convolutional Networks (KCN)" to leverage prior knowledge for CDR extraction. The proposed model first learns knowledge representations including entity embeddings and relation embeddings from KBs. Then, entity embeddings are used to control the propagation of context features towards a chemical-disease pair with gated convolutions. After that, relation embeddings are employed to further capture the weighted context features by a shared attention pooling. Finally, the weighted context features containing additional knowledge information are used for CDR extraction. Experiments on the BioCreative V CDR dataset show that the proposed KCN achieves 71.28% F1-score, which outperforms most of the state-of-the-art systems. CONCLUSIONS: This paper proposes a novel CDR extraction model KCN to make full use of prior knowledge. Experimental results demonstrate that KCN could effectively integrate prior knowledge and contexts for the performance improvement.

A new data analysis method based on feature linear combination.

In biological data, feature relationships are complex and diverse, they could reflect physiological and pathological changes. Defining simple and efficient classification rules based on feature relationships is helpful for discriminating different conditions and studying disease mechanism. The popular data analysis method, k top scoring pairs (k-TSP), explores the feature relationship by focusing on the difference of the relative level of two features in different groups and classifies samples based on the exploration. To define more efficient classification rules, we propose a new data analysis method based on the linear combination of k > 0 top scoring pairs (LC-k-TSP). LC-k-TSP applies support vector machine (SVM) to define the best linear relationship of each feature pair, scores feature pairs by the discriminative abilities of the corresponding linear combinations and selects k disjoint top scoring pairs to construct an ensemble classifier. Experiments on twelve public datasets showed the superiority of LC-k-TSP over k-TSP which evaluates the relationship of every two features in the same way. The experiment also illustrated that LC-k-TSP performed similarly to SVM and random forest (RF) in accuracy rate. LC-k-TSP studies the own unique linear combination for each feature pair and defines simple classification rules, it is easy to explore the biomedical explanation. Finally, we applied LC-k-TSP to analyze the hepatocellular carcinoma (HCC) metabolomics data and define the simple classification rules for discrimination of different liver diseases. It obtained accuracy rates of 89.76% and 89.13% in distinguishing between small HCC and hepatic cirrhosis (CIR) groups as well as between HCC and CIR groups, superior to 87.99% and 80.35% by k-TSP. Hence, defining classification rules based on feature relationships is an effective way to analyze biological data. LC-k-TSP which checks different feature pairs by their corresponding unique best linear relationship has the superiority over k-TSP which checks each pair by the same linear relationship. Availability and implementation: http://www.402.dicp.ac.cn/download_ok_4.htm.

Combining Context and Knowledge Representations for Chemical-Disease Relation Extraction.

Automatically extracting the relationships between chemicals and diseases is significantly important to various areas of biomedical research and health care. Biomedical experts have built many large-scale knowledge bases (KBs) to advance the development of biomedical research. KBs contain huge amounts of structured information about entities and relationships, therefore plays a pivotal role in chemical-disease relation (CDR) extraction. However, previous researches pay less attention to the prior knowledge existing in KBs. This paper proposes a neural network-based attention model (NAM) for CDR extraction, which makes full use of context information in documents and prior knowledge in KBs. For a pair of entities in a document, an attention mechanism is employed to select important context words with respect to the relation representations learned from KBs. Experiments on the BioCreative V CDR dataset show that combining context and knowledge representations through the attention mechanism, could significantly improve the CDR extraction performance while achieve comparable results with state-of-the-art systems.

Chemical-induced disease relation extraction with dependency information and prior knowledge.

Chemical-disease relation (CDR) extraction is significantly important to various areas of biomedical research and health care. Nowadays, many large-scale biomedical knowledge bases (KBs) containing triples about entity pairs and their relations have been built. KBs are important resources for biomedical relation extraction. However, previous research pays little attention to prior knowledge. In addition, the dependency tree contains important syntactic and semantic information, which helps to improve relation extraction. So how to effectively use it is also worth studying. In this paper, we propose a novel convolutional attention network (CAN) for CDR extraction. Firstly, we extract the shortest dependency path (SDP) between chemical and disease pairs in a sentence, which includes a sequence of words, dependency directions, and dependency relation tags. Then the convolution operations are performed on the SDP to produce deep semantic dependency features. After that, an attention mechanism is employed to learn the importance/weight of each semantic dependency vector related to knowledge representations learned from KBs. Finally, in order to combine dependency information and prior knowledge, the concatenation of weighted semantic dependency representations and knowledge representations is fed to the softmax layer for classification. Experiments on the BioCreative V CDR dataset show that our method achieves comparable performance with the state-of-the-art systems, and both dependency information and prior knowledge play important roles in CDR extraction task.

Leveraging prior knowledge for protein-protein interaction extraction with memory network.

Automatically extracting protein-protein interactions (PPIs) from biomedical literature provides additional support for precision medicine efforts. This paper proposes a novel memory network-based model (MNM) for PPI extraction, which leverages prior knowledge about protein-protein pairs with memory networks. The proposed MNM captures important context clues related to knowledge representations learned from knowledge bases. Both entity embeddings and relation embeddings of prior knowledge are effective in improving the PPI extraction model, leading to a new state-of-the-art performance on the BioCreative VI PPI dataset. The paper also shows that multiple computational layers over an external memory are superior to long short-term memory networks with the local memories.Database URL: http://www.biocreative.org/tasks/biocreative-vi/track-4/.

Exploiting syntactic and semantics information for chemical-disease relation extraction.

Identifying chemical-disease relations (CDR) from biomedical literature could improve chemical safety and toxicity studies. This article proposes a novel syntactic and semantic information exploitation method for CDR extraction. The proposed method consists of a feature-based model, a tree kernel-based model and a neural network model. The feature-based model exploits lexical features, the tree kernel-based model captures syntactic structure features, and the neural network model generates semantic representations. The motivation of our method is to fully utilize the nice properties of the three models to explore diverse information for CDR extraction. Experiments on the BioCreative V CDR dataset show that the three models are all effective for CDR extraction, and their combination could further improve extraction performance.Database URL:http://www.biocreative.org/resources/corpora/biocreative-v-cdr-corpus/.

Hedge Scope Detection in Biomedical Texts: An Effective Dependency-Based Method.

Hedge detection is used to distinguish uncertain information from facts, which is of essential importance in biomedical information extraction. The task of hedge detection is often divided into two subtasks: detecting uncertain cues and their linguistic scope. Hedge scope is a sequence of tokens including the hedge cue in a sentence. Previous hedge scope detection methods usually take all tokens in a sentence as candidate boundaries, which inevitably generate a large number of negatives for classifiers. The imbalanced instances seriously mislead classifiers and result in lower performance. This paper proposes a dependency-based candidate boundary selection method (DCBS), which selects the most likely tokens as candidate boundaries and removes the exceptional tokens which have less potential to improve the performance based on dependency tree. In addition, we employ the composite kernel to integrate lexical and syntactic information and demonstrate the effectiveness of structured syntactic features for hedge scope detection. Experiments on the CoNLL-2010 Shared Task corpus show that our method achieves 71.92% F1-score on the golden standard cues, which is 4.11% higher than the system without using DCBS. Although the candidate boundary selection method is only evaluated on hedge scope detection here, it can be popularized to other kinds of scope learning tasks.

A multistage gene normalization system integrating multiple effective methods.

Gene/protein recognition and normalization is an important preliminary step for many biological text mining tasks. In this paper, we present a multistage gene normalization system which consists of four major subtasks: pre-processing, dictionary matching, ambiguity resolution and filtering. For the first subtask, we apply the gene mention tagger developed in our earlier work, which achieves an F-score of 88.42% on the BioCreative II GM testing set. In the stage of dictionary matching, the exact matching and approximate matching between gene names and the EntrezGene lexicon have been combined. For the ambiguity resolution subtask, we propose a semantic similarity disambiguation method based on Munkres' Assignment Algorithm. At the last step, a filter based on Wikipedia has been built to remove the false positives. Experimental results show that the presented system can achieve an F-score of 90.1%, outperforming most of the state-of-the-art systems.